Nonlinear Manifold Learning Part II 6.454 Summary

Manifold learning addresses the problem of finding low–dimensional structure within collections of high–dimensional data. Recent interest in this problem was motivated by the development of a pair of algorithms, locally linear embedding (LLE) [6] and isometric feature mapping (IsoMap) [8]. Both methods use local, linear relationships to derive global, nonlinear structure, although their specific assumptions and optimization criteria differ. For an introduction to these algorithms, as well as further motivation of the manifold learning problem, see [5]. In this survey, we discuss three manifold learning algorithms which adopt the basic structure of LLE, but attempt to address some of its shortcomings. The first, Laplacian eigenmaps [1], is primarily interesting because it provides a new theoretical framework for understanding LLE. This framework points the way to the Hessian eigenmaps [4] algorithm, which explicitly attempts to estimate, and minimize, the local curvature of the embedding function. Interestingly, this Hessian extension to LLE is asymptotically correct for a strictly larger class of embeddings than any previously known algorithm. Finally, the charting algorithm [2] casts manifold learning as a density estimation problem, thereby adding robustness to noisy or sparsely sampled data.